Database sizes supported by commercially available database management systems (DBMS) continue to grow as the availability and cost per unit storage of disk-based storage and system memory increases. In general, a database can feature on-disk storage of data, in which data records are stored in one or more tables or other database structures on storage media (e.g. hard disks, optical storage, solid state storage, or the like) and read into main system memory as needed to respond to queries or other database operations. Alternatively, a database can feature in-memory storage of data, in which data records are stored in main system memory. As costs of main system memory continue to decrease, the feasibility of significant use of in-memory features increases. However, data capacity requirements of database systems also continue to increase. As such, hybrid approaches that involve features of both in-memory and on-disk systems are also advantageous.
In some examples of in-memory databases, a columnar table is composed of a delta part and a main part. The delta part receives changes to the table and stores these changes in a persistent log. Upon recovery, the delta part is rebuilt from the log. A merge process occasionally (e.g. periodically, upon the delta reaching a certain size, etc.) occasionally updates the main part. The merge process typically copies a current main state and committed rows in the delta part into a new temporary state, sorts the data for optimal compression, and overwrites the original main part. Committed rows are removed from the delta part at the time of overwriting the original main part. To prevent blocking situations during the merge process, a new delta part is typically opened as soon as the merge process starts. Uncommitted rows can be copied from the old delta part to the new delta part. Query processing can use a single delta in combination with the current main part (when a merge process is not currently in progress) or both of an old delta part and a new delta part in combination with the old main part (when a merge process is currently in progress) to execute queries.